Clinical machine learning models show a significant performance drop when tested in settings not seen during training. Domain generalisation models promise to alleviate this problem, however, there is still scepticism about whether they improve over traditional training. In this work, we take a principled approach to identifying Out of Distribution (OoD) environments, motivated by the problem of cross-hospital generalization in critical care. We propose model-based and heuristic approaches to identify OoD environments and systematically compare models with different levels of held-out information. We find that access to OoD data does not translate to increased performance, pointing to inherent limitations in defining potential OoD environments potentially due to data harmonisation and sampling. Echoing similar results with other popular clinical benchmarks in the literature, new approaches are required to evaluate robust models on health records.
translated by 谷歌翻译
深度学习算法的最新进展为解决许多医学图像分析问题带来了重大好处。培训深度学习模型通常需要具有专家标记注释的大型数据集。但是,获取专家标记的注释不仅昂贵,而且主观,容易出错,并且观察者内部变异性会引入标签。由于解剖学的模棱两可,使用深度学习模型来细分医学图像时,这尤其是一个问题。基于图像的医学诊断工具使用经过不正确分段标签训练的深度学习模型可以导致错误的诊断和治疗建议。与单评论注释相比,多评价者注释可能更适合于使用小型培训集的深度学习模型进行训练。本文的目的是开发和评估一种基于MRI中病变特征的多评价者注释和解剖学知识来生成概率标签的方法,以及一种使用概率的标签使用归一化活动性损失作为A的病变特征的解剖学知识,以训练分割模型”。耐噪声损失的功能。通过将17个膝盖MRI扫描的二进制基础真理进行比较,以评估该模型,以用于临床分割和检测骨髓病变(BML)。该方法与二进制跨透镜损失函数相比,该方法成功提高了精度14,召回22和骰子得分8%。总体而言,这项工作的结果表明,使用软标签的拟议归一化主动损失成功地减轻了嘈杂标签的影响。
translated by 谷歌翻译
抽象推理是智能系统的关键能力。大型语言模型在抽象推理任务上实现了高度的性能,但表现出许多缺陷。但是,人类的抽象推理也是不完美的,并且取决于我们对推理问题内容的知识和信念。例如,人类对在日常情况下基于逻辑规则的逻辑规则比关于抽象属性的任意规则更可靠地理解。语言模型的培训经验类似地赋予了他们先前的期望,这些期望反映了人类的知识和信念。因此,我们假设语言模型会显示出类似人类的内容对抽象推理问题的影响。我们在三个逻辑推理任务中探讨了这一假设:自然语言推论,判断三段论的逻辑有效性和ison选择任务(Wason,1968)。我们发现,最新的大语言模型(具有7或700亿个参数; Hoffman等,2022)反映了这些任务中人类在人类中观察到的许多相同模式 - 像人类一样,模型对可信情况的理由更有效地理由不现实或抽象的。我们的发现对理解这些认知效应以及有助于语言模型表现的因素具有影响。
translated by 谷歌翻译
语言模型既展示了定量的改进,又展示了新的定性功能,随着规模的增加。尽管它们具有潜在的变革性影响,但这些新能力的特征却很差。为了为未来的研究提供信息,为破坏性的新模型能力做准备,并改善社会有害的效果,至关重要的是,我们必须了解目前和近乎未来的能力和语言模型的局限性。为了应对这一挑战,我们介绍了超越模仿游戏基准(Big Bench)。 Big Bench目前由204个任务组成,由132家机构的442位作者贡献。任务主题是多样的,从语言学,儿童发展,数学,常识性推理,生物学,物理学,社会偏见,软件开发等等。 Big-Bench专注于被认为超出当前语言模型的功能的任务。我们评估了OpenAI的GPT型号,Google内部密集变压器体系结构和大型基础上的开关稀疏变压器的行为,跨越了数百万到数十亿个参数。此外,一个人类专家评估者团队执行了所有任务,以提供强大的基准。研究结果包括:模型性能和校准都随规模改善,但绝对的术语(以及与评估者的性能相比);在模型类中的性能非常相似,尽管带有稀疏性。逐渐和预测的任务通常涉及大量知识或记忆成分,而在临界规模上表现出“突破性”行为的任务通常涉及多个步骤或组成部分或脆性指标;社交偏见通常会随着含糊不清的环境而随着规模而增加,但这可以通过提示来改善。
translated by 谷歌翻译
生物医学中的多模式数据遍布,例如放射学图像和报告。大规模解释这些数据对于改善临床护理和加速临床研究至关重要。与一般领域相比,具有复杂语义的生物医学文本在视觉建模中提出了其他挑战,并且先前的工作使用了缺乏特定领域语言理解的适应性模型不足。在本文中,我们表明,有原则的文本语义建模可以大大改善自我监督的视力 - 语言处理中的对比度学习。我们发布了一种实现最先进的语言模型,从而通过改进的词汇和新颖的语言预测客观的客观利用语义和话语特征在放射学报告中获得了自然语言推断。此外,我们提出了一种自我监督的联合视觉 - 语言方法,重点是更好的文本建模。它在广泛的公开基准上建立了新的最新结果,部分是通过利用我们新的特定领域的语言模型。我们释放了一个新的数据集,该数据集具有放射科医生的局部对齐短语接地注释,以促进生物医学视觉处理中复杂语义建模的研究。广泛的评估,包括在此新数据集中,表明我们的对比学习方法在文本语义建模的帮助下,尽管仅使用了全球对准目标,但在细分任务中的表现都优于细分任务中的先验方法。
translated by 谷歌翻译
解释在人类学习中发挥着相当大的作用,特别是在仍然在形成抽象的主要挑战,以及了解世界的关系和因果结构的地区。在这里,我们探索强化学习代理人是否同样可以从解释中受益。我们概述了一系列关系任务,涉及选择一个在一个集合中奇数一个的对象(即,沿许多可能的特征尺寸之一的唯一)。奇数一张任务要求代理在一组对象中的多维关系上推理。我们展示了代理商不会仅从奖励中学习这些任务,但是当它们也培训以生成语言解释对象属性或选择正确或不正确时,实现> 90%的性能。在进一步的实验中,我们展示了预测的解释如何使代理能够从模糊,因果困难的训练中适当地推广,甚至可以学习执行实验干预以识别因果结构。我们表明解释有助于克服代理人来解决简单特征的趋势,并探讨解释的哪些方面使它们成为最有益的。我们的结果表明,从解释中学习是一种强大的原则,可以为培训更强大和一般机器学习系统提供有希望的道路。
translated by 谷歌翻译
具有功能近红外光谱(FNIR)的高度催化是一种新兴的神经影像应用,可以测量社交互动的差异性神经签名。研究人员评估了性别和任务类型(例如,合作与竞争)对人类互动期间脑内一致性的影响。然而,没有工作尚未使用基于深入的学习方法,以提取对Fnirs Hyperscanning背景中的性别和基于任务的差异的洞察。这项工作提出了一种基于卷积神经网络的基于神经网络的Dyadic Dex Contement和任务分类方法,以获得N = 222美元参与者的广泛的超阳扫描数据集。使用动态时间翘曲计算的脑间信号相似度用作输入数据。该拟议的方法实现了大于80美元的最大分类准确性,从而为探索和理解复杂的大脑行为提供了新的大道。
translated by 谷歌翻译
While the brain connectivity network can inform the understanding and diagnosis of developmental dyslexia, its cause-effect relationships have not yet enough been examined. Employing electroencephalography signals and band-limited white noise stimulus at 4.8 Hz (prosodic-syllabic frequency), we measure the phase Granger causalities among channels to identify differences between dyslexic learners and controls, thereby proposing a method to calculate directional connectivity. As causal relationships run in both directions, we explore three scenarios, namely channels' activity as sources, as sinks, and in total. Our proposed method can be used for both classification and exploratory analysis. In all scenarios, we find confirmation of the established right-lateralized Theta sampling network anomaly, in line with the temporal sampling framework's assumption of oscillatory differences in the Theta and Gamma bands. Further, we show that this anomaly primarily occurs in the causal relationships of channels acting as sinks, where it is significantly more pronounced than when only total activity is observed. In the sink scenario, our classifier obtains 0.84 and 0.88 accuracy and 0.87 and 0.93 AUC for the Theta and Gamma bands, respectively.
translated by 谷歌翻译
There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/ .
translated by 谷歌翻译
The ability to convert reciprocating, i.e., alternating, actuation into rotary motion using linkages is hindered fundamentally by their poor torque transmission capability around kinematic singularity configurations. Here, we harness the elastic potential energy of a linear spring attached to the coupler link of four-bar mechanisms to manipulate force transmission around the kinematic singularities. We developed a theoretical model to explore the parameter space for proper force transmission in slider-crank and rocker-crank four-bar kinematics. Finally, we verified the proposed model and methodology by building and testing a macro-scale prototype of a slider-crank mechanism. We expect this approach to enable the development of small-scale rotary engines and robotic devices with closed kinematic chains dealing with serial kinematic singularities, such as linkages and parallel manipulators.
translated by 谷歌翻译